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Abstract: We discuss similarities and differences between systems of interacting players max- 
imizing their individual payoffs and particles minimizing their interaction energy. Long-run 
behavior of stochastic dynamics of spatial games with multiple Nash equilibria is analyzed. In 
particular, we construct an example of a spatial game with three strategies, where stochastic 
stability of Nash equilibria depends on the number of players and the kind of dynamics. 

KEY WORDS: Evolutionary game theory; Nash equilibria; cellular automata; stochastic 
stability. 



1 



1 Introduction 



Many socio-economic systems can be modeled as systems of interacting individuals; see for ex- 
ample Santa Fe collection of papers on economic complex systems [1] and econophysics bulletin 
[2] . One may then try to derive their global behavior from individual interactions between their 
basic entities. Such approach is fundamental in statistical physics which deals with systems of 
interacting particles. We will explore similarities and differences between systems of interacting 
players maximizing their individual payoffs and particles minimizing their interaction energy. 

Here we will consider game-theoretic models of many interacting agents [3, 4, 5]. In such 
models, agents have at their disposal certain strategies and their payoffs in a game depend on 
strategies chosen both by them and by their opponents. In spatial games, agents are located 
on vertices of certain graphs and they interact only with their neighbors [6, 7, 8, 9, 11, 12, 
13, 14, 15, 16]. The central concept in game theory is that of a Nash equilibrium. It is an 
assignment of strategies to players such that no player, for fixed strategies of his opponents, 
has an incentive to deviate from his curent strategy; the change can only diminish his payoff. 

One of the best known game is that of a Prisoner's Dilemma game [10]. It has a unique 
Nash equilibrium, when both players defect. In fact, defection is the best response to both 
cooperation and defection of the opponent. However, both players are better off when they 
cooperate. Dynamical aspects of spatial prisoner's dillema games were discussed in many 
papers, see for example [11, 12, 13, 14, 15, 16]. Players in these games adapt to their environment 
by imitating those with biggest payoffs. It was shown that cooperation persists for a certain 
range of parameters. 

Here we will discuss games with multiple Nash equilibria. One of the fundamental problems 
in game theory is the equilibrium selection in such games. One of the selection methods is 
to construct a dynamical system where in the long run only one equilibrium is played with a 
high frequency. John Maynard Smith [17, 18] has refined the concept of equilibrium to include 
the stability of Nash equilibria against mutants. He introduced the fundamental notion of 
an evolutionarily stable strategy. If everybody plays such a strategy, then the small number 
of mutants playing a different strategy is eliminated from the population. The dynamical 
interpretation of the evolutionarily stable strategy was later provided by several authors [19, 
20, 21]. They proposed a system of differential replicator equations, which describe the time- 
evolution of frequencies of strategies and analyzed the asymptotic stability of Nash equilibria. 

Here we will discuss a stochastic adaptation dynamics of a population with a fixed number 
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of players. In discrete moments of times, players adapt to their neighbors by choosing with a 
high probability the strategy which is the best response, i.e. the one which maximizes the sum 
of the payoffs obtained from individual games. With a small probability, representing the noise 
of the system, they make mistakes. We study the long-time behavior of such dynamics. We say 
that a configuration of strategies is stochastically stable [22] if it has a positive probability 
in the stationary state of the above dynamics in the zero- noise limit, that is zero probability of 
mistakes. It means that in the long run we observe it with a positive frequency. 

In Section 2, we introduce basic notions of game theory and discuss similarities and dif- 
ferences between ground-state configurations in classical lattice-gas models and Nash config- 
urations in game theory. Section 3 contains the description of a simple stochastic dynamics. 
In Section 4, we present an example of a spatial game with three strategies, where stochas- 
tic stability of Nash equilibria depends on the number of players and the kind of dynamics. 
Discussion follows in Section 5. 

2 Nash configurations 

To characterize a game-theoretic model one has to specify the set of players, strategies they 
have at their disposal and payoffs they receive. Although in many models the number of 
players is very large, their strategic interactions are usually decomposed into a sum of two- 
player games. Only recently, there have appeared some systematic studies of truly multi-player 
games [23, 24, 25]. Here we will discuss only two-player games with two or three strategies. We 
begin with games with two strategies. A generic payoff matrix is given by 
Example 1 

A B 

A a b 

U = 

Bed, 

where the ij entry, i,j = A,B, is the payoff of the first (row) player when he plays the 
strategy i and the second (column) player plays the strategy j. We assume that both players 
are the same and hence payoffs of the column player are given by the matrix transposed to U ; 
such games are called symmetric. 

An assignment of strategies to both players is a Nash equilibrium, if for each player, for a 
fixed strategy of his opponent, changing the current strategy will not increase his payoff. If 
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c> a,d > b and a > d, then the game has a unique Nash equilibrium (B, B) but both players 
are much better off when they play A - this is the classic Prisoner's dilemma case [10]. If a < c 
and d < b, then there are two nonsymmetric Nash equilibria: (A, B) and (B, A) (a Hawk-Dove 
game [18]). Below we will discuss games with multiple Nash equilibria. If a > c and d > b, 
then both (A, A) and (B, B) are Nash equilibria. If a + b < c + d, then the strategy B has a 
higher expected payoff against a player playing both strategies with the probability 1/2. We 
say that B risk dominates the strategy A (the notion of the risk-dominance was introduced and 
thoroughly studied by Harsanyi and Selten [26]). If at the same time a > d, then we have a 
selection problem of choosing between the payoff-dominant (Pareto-efficient) equilibrium (A, A) 
and the risk-dominant (B,B). 

Let us now describe spatial games with local interactions. Let A be a finite subset of the 
simple lattice Z d . Every site of A is occupied by one player who has at his disposal one of k 
different strategies (k — 2 in the above example). Let S be the set of strategies, then il\ = S A 
is the space of all possible configurations of players, that is all possible assignments of strategies 
to individual players. For every % E A, X« is the strategy of the %— th player in the configuration 
X E and X_j denotes strategies of all remaining players; X therefore can be represented 
as the pair (Xj,X_j). Every player interacts only with his nearest neighbors and his payoff 
is the sum of the payoffs resulting from individual plays. We assume that he has to use the 
same strategy for all neighbors. Let iVj denote the neighborhood of the i— th player. For the 
nearest-neighbor interaction we have iVj = {j; \j — — 1}, where \i — j\ is the distance between 
% and j. For I 6 fl A we denote by Vi(X) the payoff of the %— th player in the configuration X: 

Vi (X)= £ U(X i: Xj), (1) 

where U is a k x k matrix of payoffs of a two-player symmetric game with k strategies. 
Definition 1 X E £l A is a Nash configuration if for every i E A and Y { E S, u^X^X^) > 

In Example 1 we have two homogeneous Nash configurations, X A and X B , in which all 
players play the same strategy, A or B respectively. 

Let us notice that the notion of a Nash configuration is similar to the notion of a ground-state 
configuration in classical lattice-gas models of interacting particles. We have to identify agents 
with particles, strategies with types of particles and instead of maximizing payoffs we should 
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minimize interaction energies. There are however profound differences. First of all, ground-state 
configurations can be defined only for symmetric matrices; an interaction energy is assigned to 
a pair of particles, payoffs are assigned to individual players and may be different for each of 
them. In fact, it may happen that if a player switches a strategy to increase his payoff, the 
payoff of his opponent and of the entire population decreases (like in Prisoner's Dilemma game). 
Moreover, ground-state configurations are stable with respect to all local changes, not just one- 
site changes like Nash configurations. It means that for the same symmetric matrix U, there 
may exist a configuration which is a Nash configuration but not a ground-state configuration for 
the interaction marix —U. The simplest example is given by Example 1 with a = 2, b = c = 0, 
and d — 1. X A and X B are Nash configurations but only X A is a ground-state configuration 
for -U. 

Games with symmetric payoff matrices are called doubly symmetric or potential games [27]. 

More generally, a game is called a potential game if its payoff matrix can be changed to 
a symmetric one by adding payoffs to its columns. As we know, such a payoff transformation 
does not change strategic character of the game, in particular it does not change the set of 
its Nash equilibria. More formally, it means that there exists a symmetric matrix V called a 
potential of the game such that for any three strategies A, B, C G S 

U(A, C) - U(B, C) = V(A, C) - V(B, C). (2) 

It is easy to see that every game with two strategies has a potential V with V(A, A) = a — c, 
V(B, B) = d — b, and V(A, B) = V(B, A) = 0. It follows that an equilibrium is risk-dominant 
if and only if it has a bigger potential. 

For players on a lattice, for any X e fU, 

V(X)= £ V{X i ,X j ) (3) 

(*J)CA 

is then a potential of the configuration X. 

For any classical lattice-gas model there exists at least one ground-state configuration. This 
can be seen in the following way. We start with an arbitrary configuration. If it cannot be 
changed locally to decrease its energy it is already a ground-state configuration. Otherwise we 
may change it locally and decrease the energy of the system. If our system is finite, then after 
a finite number of steps we arrive at a ground-state configuration; at every step we decrease 
the energy of the system and for every finite system its possible energies form a finite set. For 
an infinite system, we have to proceed ad infinitum converging to a ground-state configuration 
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(this follows from the compactness of S z2 ). Game models are different. It may happen that a 
game with a nonsymmetric payoff matrix may not posess a Nash configuration. The classical 
example is that of the Rock-Scissors-Paper game given by the following matrix. 



Example 2 





R 


S 


P 


R 


1 


2 





S 





1 


2 


P 


2 





1 



One may show that this game does not have any Nash configurations on Z and Z 2 but many 
Nash configurations on the triangular lattice. 

In short, ground-state configurations minimize the total energy of a particle system, Nash 
configurations do not necessarily maximize the total payoff of a population of agents. 



3 Stochastic Stability 

We describe now the deterministic dynamics of the best-response rule. Namely, at each 
discrete moment of time t = 1,2,..., a randomly chosen player may update his strategy. He 
simply adopts the strategy, X\, which gives him the maximal total payoff Vi(X\, X 1 ^ 1 ) for given 
Xti 1 , a configuration of strategies of remaining players at the time t — 1. 

Now we allow players to make mistakes with a small probability, that is to say they may 
not choose best responses. We will discuss two types of such stochastic dynamics. In the first 
one, the so-called perturbed best response, a player follows the best-response rule with 
probability 1 — e (in case of more than one best-response strategy he chooses randomly one of 
them) and with probability e he makes a "mistake" and chooses randomly one of the remaining 
strategies. The probability of mistakes (or the noise level) is state-independent here. 

In the log-linear rule, the probability of chosing by the %— th player the strategy X\ at the 
time t decreases with the loss of the payoff and is given by the following conditional probability: 

Pl{Xt\X^) = T-^r y (4) 

Let us observe that if e — * 0, p\ converges pointwise to the best-response rule. Both 
stochastic dynamics are examples of irreducible Markov chains (there is a nonzero probability 
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to move from any state to any other state in a finite number of steps) with \ S A \ states. Therefore, 
they have unique stationary probability distributions denoted by fj, e A . 
The following definition was introduced by Foster and Young [22]: 

Definition 2 X £ il A is stochastically stable if \im e ^ /j, € A (X) > 0. 

If X is stochastically stable, then the frequency of visiting X converges to a positive number 
along any time trajectory almost surely. It means that in the long run we observe X with a 
positive frequency. 

Stationary distributions of log-linear dynamics can be explicitly constructed for potential 
games. It can be shown [8] that the stationary distribution of the log-linear dynamics in a game 
with the potential V is given by 

= -T^-y (5) 

We may now explicitly perform the limit e — > in (5). In Example 1, X B has the biggest 
potential (which is equivalent to the risk dominance of B) so lim^o n A (X B ) = 1 hence X B is 
stochastically stable (we also say that B is stochastically stable). 

Let us now consider coordination games with three strategies and three symmetric Nash 
equilibria: (A, A), (B, B), and (C, C). One may say that A risk dominates the other two 
strategies if it risk dominates them in pairwise comparisons. Of course it may happen that 
A dominates B, B dominates C, and finally C dominates A. But even if we do not have 
such a cyclic relation of dominance, a strategy which is pairwise risk-dominant may not be 
stochastically stable as we will see below. A more relevant notion seems to be that of a global 
risk dominance [28]. We say that A is globally risk dominant if it provides a maximal payoff 
against a mixed strategy (a probability distribution on strategies) which assigns the probability 
1/2 to A. It was shown that a globally risk-dominant strategy is stochastically stable for some 
spatial games with nearest-neighbor interactions [7, 9]. A different criterion for stochastic 
stability was developed by Blume [6] . He showed (using methods of statistical mechanics) that 
in games with k strategies A iy i = 1, k and k symmetric Nash equilibria, A 1 is stochastically 
stable if 

min(C/(A 1 , A ± ) - U(A n , AJ) > max(U(A n , A n ) - U(A U A n )). (6) 

n>\ n>l 

We may observe that if Ai satisfies the above condition, then it is pairwise risk dominant. 
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4 Example 



Let us now present our example of a game with three strategies. Players are located on a finite 
subset of the one-dimensional lattice Z and interact with their nearest neighbors only. Denote 
by n the number of players, For simplicity we will assume periodic boundary conditions, that 
is we will identify the n + 1-th player with the first one. In other words, the players are located 
on the circle. 

The payoffs are given by the following matrix: 
Example 3 

ABC 
A 1 + a 1.5 
U = B 2 
C 3 

with a < 0.5. 

As before, we have three homogeneous Nash configurations, X A , X B , and X° . We will consider 
here both the log-linear and perturbed best-response dynamics. The game is not a potential 
one so there is no explicit formula for the stationary distribution. 

To find stochastically stable states, we must resort to different methods. We will use a 
tree representation of the stationary distribution of Markov chains [29, 30, 31]. Let (£l,P) 
be an irreducible Markov chain with a state space Q and transition probabilities given by 
P : x — > [0, 1]. It has a unique stationary distribution. A stationary distribution is an 
eigenvector of the transition matrix P corresponding to the eigenvalue 1, i.e., a solution of a 
system of linear equations. After a specific rearrangement one can arrive at an expression for 
the stationary state which involves only positive terms. This will be very usefull in describing 
asymptotic behavior of a stationary state. 

For X G f2, let X-tree be a directed graph on f2 such that from every Y ^ X there is a 
unique path to X and there are no outcoming edges out of X. Denote by T(X) the set of all 
X-trees and let 

«po= e n p ( Y > Y ')> (7) 

deT(X) (Y,Y')ed 

where the product is with respect to all edges of d. 

The following representation of the stationary distribution ji was provided by Freidlin and 
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Wentzell in [29, 30] (cf also[31]): 
for all iGfl. 

The above characterisation of the stationary distribution was used to find stochastically 
states in nonspatial [32, 33] and spatial games [7, 9]. Here we will apply it for our nonpotential 
game. 

Let us note that X A , X B , and X c are the only absorbing states of the noise- free dynamics. 
When we start with any state different from X A , X B , and X c , then after a finite number of 
steps of the best-response dynamics we arrive at either X A , X s or X c and then stay there 
forever. It follows from the tree representation of the stationary distribution that any state 
different from absorbing states has zero probability in the stationary distribution in the zero- 
noise limit. Moreover, in order to study the zero- noise limit of the stationary distribution, it is 
enough to consider paths between absorbing states. More precisely, we construct X-trees with 
absorbing states as vertices; the family of such X-trees is denoted by T{X). Let 

q m (X) = max def(x) J] P(Y,Y% (9) 

(Y,Y')ed 

where P(Y, Y') = max ri(w/w/') P{W, W), where the product is taken along any path joining Y 
with Y' and the maximum is taken with respect to all such paths. Now we may observe that if 
lim e ^oq m (X l ) / q m (X ) = 0, i — A, B, then X c is stochastically stable. Therefore we have to 
compare trees with the biggest products in (9); such trees we call maximal. 

We begin with a stochastic dynamics with a state-independent noise. Let us consider the 
case of a < 0.5. It is easy to see that q m (X c ) is of order e 2 , q m (X B ) is of order e n , and q m (X A ) 
is of order e 2 ^ -1 ). We obtained the following theorem. 

Theorem 1 If a < 0.5, then X c is stochastically stable in the perturbed best-response dynam- 
ics. 

Let us now consider the log- linear rule. 

Theorem 2 Ifn < 2 + 1/(0.5 — a), then X s is stochastically stable and if n > 2 + 1/(0.5 — a), 
then X c is stochastically stable in the log-linear dynamics. 

Proof: The following are maximal A-tree, B-tree, and C-tree: 

B —> C —> A, C^A^B, A^B^C, 
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where the probability of A — > B is equal to 



1 ( - ) n ~ 2 - (10) 



1 + 1 + e/3(2+2a) V X + e -2/3 + e /3(-l+a) ' ]_ + g-4/3 + g-4/3 ' 

the probability of 5 — > C is equal to 

l + l + e^ K l + e-P + e~ 1 - 5 P ) 1 + e'W + e~^ ' 1 ; 

and the probability of C — > A is equal to 



1 , 1 N n-2 1 



(12) 



1 _|_ e -W + e 3/3 v 1 _|_ e -/3(2.5+a) + g/3(0.5-a) ' X + g-2/3(l+a) + g-2/3(l+a) 

Let us observe that 

Pb^a = 0(e-^°- 5 - a ^ n -^), (13) 

P C ^ B = o( e ^( 5 + 2 «+( - 5 -«)(«- 2 ))), (14) 

JW*7 = 0(e^ 6+2 «)), (15) 

where (3 — 1/e and linx^o 0{x)/x = 1. 
Now if n < 2 + 1/(0.5 - a), then 

lim^4^ = lim^^ = (16) 

^Og m (X B ) -0P C ^ B 1 J 

which finishes the proof. 

It follows that for small enough n, X B is stochastically stable and for big enough n, X c 
is stochastically stable. We see that adding more players to the population may change the 
stochastic stability of Nash configurations. Let us also notice that the strategy C is globally 
risk dominant. Nevertheless, it is not stochastically stable in the log-linear dynamics for a 
sufficiently small number of players. 

Let us now discuss the case of a — 0.5. 

Theorem 3 If a = 0.5, then X B is stochastically stable for any n in the log-linear dynamics. 
Proof: 



X is stochastically stable which means that for any fixed number of players, if the noise is 
sufficiently small, then in the long run we observe B players with an arbitrarily high frequency. 
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However, for any low but fixed noise, if the number of players is big enough, the probability 
of any individual configuration is practically zero. It may happen though that the stationary 
distribution is highly concentrated on an ensemble consisting of one Nash configuration and 
its small perturbations, i.e. configurations, where most players play the same strategy. We 
will call such configurations ensemble stable [34]. An ensemble-stable configuration may not 
be stochastically stable. In fact, we expect that X c is ensemble stable because its lowest-cost 
excitations occur with a probability of order e _3/3 and those from X B with a probability of 
order e~ 4/3 . We observe this in simple Monte-Carlo simulations. 

5 Discussion 

We studied the problem of equilibrium selection in spatial games with many players. We 
showed that stochastic stability of Nash configurations may depend on the number of players. 
In particular, we presented an example with the globally risk dominant equilibrium which is 
stochastically stable in the perturbed best response dynamics and is not stochastically stable 
in the log-linear one if the number of players is smaller than some critical value which grows to 
infinity when some payoff parameter approaches a critical value. 
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